Papers with reduced robustness
Comprehensiveness Metrics for Automatic Evaluation of Factual Recall in Text Generation (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) produce incomplete or selectively omit key information . omissions of key information or misrepresentation of conflicting evidence can cause harm . |
| Approach: | They propose a method that decomposes texts into atomic statements and uses natural language inference to identify missing facts and a Q A-based metric that extracts question-answer pairs and compares responses across sources. |
| Outcome: | The proposed evaluation metrics show they perform better than more complex metrics, but at a cost. |